\documentclass[letterpaper, twocolumn]{article}
\usepackage[left=1in,top=1in,right=1in,bottom=1in,nohead]{geometry}
\usepackage{booktabs}


\author{Daniel L. Wang}
\title{The SWAMP Quick Reference}
\begin{document}
\maketitle
\section{Introduction}
The Script Workflow Analysis for MultiProcessing(SWAMP) system is
designed to help everyday scientists analyze their own and each
others' data.  SWAMP \emph{server} instances provide computational
service to complement simple data publishing.  A lightweight
\emph{client} program manages user shell script submission and results
download as simply as possible.

This quick-reference describes client-perspective operation.
\subsection{Quickstart}
Download the \emph{swamp-client}  of SWAMP from the SWAMP website at
http://swamp.googlecode.com , and try running things as described in
the Quickstart page \footnote{
http://code.google.com/p/swamp/wiki/SwampQuickstart}.


\subsection{Basic Idea}
\small
\begin{verbatim}
export SWAMPURL='http://pbs.ess.uci.edu:8080/SOAP'
python swamp_client.py ipcctest.swamp 
\end{verbatim}
\normalsize
The first line declares the default SWAMP service to contact.  You can
think of this as symbolizing the place where your data and computation
are hosted, as far as SWAMP is concerned.

The second line submits the script ipcctest.swamp to the default
SWAMP service, waits for computation to complete, and downloads the
results.  This works like \texttt{qsub}, except that it waits for
things to finish, and downloads results.

The same standard \texttt{sh} shell
script that runs your personal sequence of NCO commands can be sent to
SWAMP mostly unmodified, aside from path changes that can be switched
using \emph{if}-statements.  SWAMP takes care of the hard work, like
figuring out which files are input and output, and how to execute
things in an efficient manner. 

\section{Shell Syntax}
Details can be found in Appendix B of my thesis.  Some highlights:
\paragraph{Variables} Although there are two types of variables to
consider in a normal shell environment (shell and environment), SWAMP
considers them equal and isolated.  Scripts cannot see environment
variables of their running hosts, nor are user shell or environment
variables sent over with the script.  Scripts should set and use
whatever variables they need within the script body.  

\paragraph{Basic program invocation for NCO programs}
NCO commands supported are: \texttt{ncap}, \texttt{ncap2}\footnote{
10/24/08: ncap2 support will be added before I leave.},
\texttt{ncatted},  
\texttt{ncbo}, \texttt{ncdiff}, \texttt{ncea}, \texttt{ncecat},
\texttt{ncflint}, \texttt{ncks}, \texttt{ncpack}, \texttt{ncpdq},
\texttt{ncra}, \texttt{ncrcat}, \texttt{ncrename}, \texttt{ncunpack},
\texttt{ncwa}. 

Example: \texttt{ ncwa \$\{CASEID\}.\$\{month\}.series.nc \$\{CASEID\}.\$\{month\}.avg.nc}

Don't put the path to the commands.  No: \texttt{/usr/bin/ncwa} Yes: \texttt{ncwa}.
\paragraph{Wildcards}  Use wildcards freely to match against files in
a directory\footnote{This works on files generated during the script,
  but not files generated by other scripts.}
\paragraph{Loops} The standard 
\begin{verbatim}
for i in 01 02  ; do
   ncra cam2.h0.????-$i.nc cam2_clm$i.nc
done
\end{verbatim}
loop syntax is supported. 
\paragraph{Backtick Expressions}
Only \texttt{printf} and \texttt{seq} are supported, for safety reasons.
\begin{verbatim}
START='01'
END='10'
for yra in `seq $START ${END}` ; do 
  yr=`printf "%02d" ${yra}`
  ncwa ccsm${yr}.nc avg${yr}.nc
done
\end{verbatim}
Ask me if you need more.

\paragraph{Conditional Statements}
\begin{verbatim}
typ=1  # 0=test
prd=1 #
if [ "${typ}" = '0' ] ; then
    ncra -O ~/nco/data/in.nc ~/foo.nc
elif [ "${typ}" = "${prd}" ] ; then 
    for yra in `seq 1 4` `seq 5 12` ; do
        yr=`printf "%02d" ${yra}`
        ncra -O ${yr}.nc foo${yr}.nc
    done
fi # !prd
\end{verbatim}
=, ==, $<$ and $>$ are supported.  Conditionals must not depend on
results of NCO programs within the script.

\paragraph{Special Globals}
To help make scripts automatically portable with SWAMP, the following
variables may be used to detect SWAMP: \emph{SWAMPVERSION},
\emph{SWAMPHOSTNAME}, and \emph{SHELL}\footnote{SHELL is a standard
  variable available in \texttt{sh} and \texttt{bash}.}.

\paragraph{Unsupported/Future syntax}
There isn't support for \texttt{exit} calls.  Nor is there support for
automatic handling of multi-file scripts (whose logic is spread among
multiple files).  Command-line arguments are not passed to the script,
though this feature is planned.

\section{Recommended Usage}
The best way to use SWAMP is to use it with the same scripts you
already have.  Make sure your scripts do the right thing on small bits
of data, and then put in conditional statements so they run on big
datasets in a SWAMP environment.

\section{Notes}

\paragraph{Isolation} Execution is isolated between scripts.  Don't
worry about leftover files from past script invokations--they won't
interfere.

\paragraph{``Multi-file'' Scripts} Scripted analyses are often
scripted in multiple files (i.e. one shell script, multiple ncap/ncap2
scripts).  SWAMP doesn't currently detect these and submit them
automatically--ask your sysadmin how to upload these files.



\end{document}
